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Non-linear Least Squares Fitting in IDL with MPFIT 



Craig B. Markwardt-*^- 

Abstract. MPFIT is a port to IDL of the non-linear least squares fitting pro- 
gram MINPACK-I. MPFIT inherits the robustness of the original FORTRAN 
version of MINPACK-1, but is optimized for performance and convenience in 
IDL. In addition to the main fitting engine, MPFIT, several specialized functions 
are provided to fit 1-D curves and 2-D images; I-D and 2-D peaks; and interac- 
tive fitting from the IDL command line. Several constraints can be applied to 
model parameters, including fixed constraints, simple bounding constraints, and 
"tying" the value to another parameter. Several data weighting methods are al- 
lowed, and the parameter covariance matrix is computed. Extensive diagnostic 
capabilities are available during the fit, via a call-back subroutine, and after the 
fit is complete. Several different forms of documentation are provided, including 
a tutorial, reference pages, and frequently asked questions. The package has 
been translated to C and Python as well. The full IDL and C packages can be 
found at ht tp : // purl . com/ net / mpf it[ 



1. Introduction 

Non-linear least squares fitting is an integral part of most astronomical anal- 
ysis. The process embodies the fundamental process of hypothesis testing for 
a candidate model which may explain the data. There are several built-in fit- 
ting procedures packaged within the Interactive Data Language (IDL) produciQ- 
Unfortunately, the existing IDL procedures are not very desirable from the per- 
spective of astronomical data analysis. The built-in procedures CURVEFIT and 
LMFIT are somewhat unreliable, and do not always take advantage of IDL's vec- 
torization capability. Because of these limitations, the author undertook to write 
a robust and functional least squares fitting code for IDL. The work was based 
on translating the highly successful MINPACK-1 package written in FORTRAN 
into IDL, and building new functionality upon that framework. 

2. The Heritage of MINPACK 

MPFIT is basically a translation and enhancement of the MINPACK-1 software, 
originally developed by Jose More collaborators at Argonne National Labora- 
tories. The code was written in FORTRAN, and is available now from the 
NETLIB software repository. MINPACK-1 has the advantages that it is: 



^Department of Astronomy, University of Maryland, College Park, MD, USA 
^CRESST and Astroparticle Physics Laboratory, NASA/GSFC, Greenbelt, MD, USA 
^IDL is a product of ITT Visual Information Solutions, jhttp : //ittvis . com/| 

1 



2 



Markwardt 



• robust — designed by numerical analysts with real data in mind 

• self-contained — not dependent on a large external library 

• general — capable of solving most non-linear equations 

• well-known — one of the most-used libraries in optimization problems 
The original MINPACK-1 library contains two different versions, Imder and 
Imdif . Both require the user function to compute the residual vector, r, but 
Imder also requires the user to compute the Jacobian matrix, J, of the residuals 
as well; Imdif estimates the Jacobian via finite differences. The MINPACK 
algorithm solves the problem by linearizing it around the trial parameter set, 
Po, and solving for an improved parameter set, p = po + 6p, via the least squares 
equation, {j'^ J)6p = —J^r. The solution is obtained by QR factorization of J, 
leading to improved numerical accuracy over the normal equations form. The 
standard Levenberg-Marquardt technique of replacing the first parenthesized 
term with [J^ J + XD"^ D), where A is the Levenberg Marquardt parameter and 
L> is a diagonal scaling matrix, produces faster convergence. The solution is 
iterated until user-selected convergence criteria are achieved, based on the sum 
of squares and residual values. 



3. Translation to IDL 

The translation to IDL focused on preserving the quality of the original code, 
optimizing it for speed within IDL, and adding functionality within the semantics 
of IDL. The result of the translation is a single fitting engine, MPFIT, which 
provides all of the original MINPACK-1 capability. This function is not specific 
to a particular problem, i.e. it can be used on data of arbitrary dimension or 
weighting. 

In addition to the generic fitting routines, several convenience routines have 
been developed that make MPFIT useful in several specific problem domains: 

• MPFITFUN, MPFIT2DFUN — optimized for 1-D k 2-D functions; 

• MPFITEXPR — for dynamically-created formulae, e.g. on the command line; 

• MPCURVEFIT — a drop-in replacement for the standard CURVEFIT IDL li- 
brary routine, for users who need compatibility; 

• MPFITPEAK, MPFIT2DPEAK — speciahzed for 1-D & 2-D peak fitting; 

• MPFITELLIPSE — for fitting elliptical curves to X/Y scatter points. 
The IDL version can be found on the author's website (see Resources, sec. [ST]) . 



4. Innovations of MPFIT 

Beyond the original MINPACK-1 code, MPFIT contains several innovations 
which enhances its usefulness and convenience to the user, and also take advan- 
tage of the capabilities of IDL. 

Private Data. The user can pass any private data safely to the user 
function as keyword variables via the FUNCTARGS parameter. This helps to avoid 
the use of common block variables. 

Parameter Constraints. The notion of simple parameter boundary con- 
straints is supported via the PARINFO parameter. Individually settable upper 
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and lower limits are supported via LIMITS. Also, as a convenience, parameters 
can be held FIXED, or TIED to another parameter value. 

The total number of degrees of freedom is tracked, as well as the number 
of parameters pegged at their limits (via the DOF and NPEGGED keywords). 

Jacobian Calculations. The user is free to supply explicit derivatives in 
their user function, or have MPFIT calculate them numerically, depending on 
the AUTODERIVATIVE and PARINFO.MPSIDE settings. The method for calculation 
of derivatives (step size and direction) are settable on a per-parameter basis via 
the PARINFO.STEP and .RELSTEP settings. For user-calculated derivatives, the 
user can enable a debugging mode by setting PARINFO .MPDERIV_DEBUG. 

Covariance matrix. The capability to calculate the covariance matrix of 
the fit parameters is an improvement over the original published MINPACK-1 
version. 

Hard-to-Compute Functions. For functions that are difficult to com- 
pute within a single function call, MPFIT can be requested to allow 'external' 
evaluation. MPFIT then returns control temporarily to the caller so that it can 
compute the function using external information and by whatever means, and 
then the caller re-calls MPFIT to resume fitting. 

Iteration Function. After each iteration, a user procedure designated 
by ITERPROC may be called. The default procedure simply prints the parameter 
values, but a more advanced version may be used, for example for GUI feedback. 

Error handling. Two error status parameters are provided. Upon return, 
STATUS is set to a numerical status code suitable for automated response. ERRMSG 
is set to a descriptive error string to inform the human user of the problem. 
MPFIT also traps common problems, like user- function errors and numerical 
over / under- flows . 

5. Documentation 

MPFIT is provided with extensive documentation. The MPFIT source code 
has reference-style documentation attached to the header of the source module 
itself. 

A basic tutorial is provided on the author's web page (see sec. [8?]), which 
introduces the user to least squares fitting of a 1-D data set, and graduates to 
applying parameter constraints. Also, a 'FAQ' style web page gives users quick 
answers to common questions, such as which module to use, how to calculate 
important quantities, and troubleshooting techniques. 

6. Usage 

Examples of usage can be found on the author's website, and as a part of the 
code documentation itself. As an example, consider a user that has a data set 
with independent variable X and dependent variable Y (with Gaussian errors EY), 
and wants to fit as a function of F(X,P) where P is an array of parameters. 

In this case, MPFITFUN should be used to solve for the best fit parameters 
PBEST with the following invocation, 
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PBEST = MPFITFUN ( ' F ' , X, Y, EY, PSTART, STATUS=ST, ERRMSG=ERR, $ 
BESTN0RM=CHI2, DOF=DOF, ERRDR=PERRDR , CDVAR=CDVAR) 

where PSTART is an initial guess of the parameter values. Upon return, the 
best fit value and degrees of freedom are returned via the BESTNORM and DOF 
parameters. Parameter errors and covariance matrix are returned in the ERROR 
and COVAR parameters. Error conditions are returned in STATUS and ERRMSG. 

7. Results 

MPFIT has been available for ten years from the author's web site, and as been 
downloaded several thousand times. During that time, the package has been 
continuously improved, both in terms of functionality, and in terms of fixing 
"bugs." By its nature, IDL code is "open source," and at least ten users have 
contributed changes which have been incorporated into the main code base. 
MPFIT is distributed with very liberal licensing constraints. 

The package has been acknowledged as helpful in a number of published 
works, including at least 29 refereed publications since 2001 (including Astro- 
physical Journal, Monthly Notices and PAS J), and in 102 preprints on the Arxiv 
preprint server. Interestingly, MPFIT has also been translated into the Python 
language, and is available in the SciPy scientific package (the interesting aspect 
is that the translation was based on the IDL version and not the original FOR- 
TRAN). The author has also create a C translation of MPFIT, which has the 
benefit of speed and portability, along with many of the IDL-based improve- 
ments. 

In addition to being used in scientific analysis, MPFIT has also been incor- 
porated into numerous standalone packages, for example PAN ("PEAK Analy- 
sis" ) for neutron scattering spectroscopy, and PintOfAle for X-ray spectroscopy. 

8. Resources 

• MPFIT IDL & C code: http : / /purl . com/net/mpf it 

• MPFIT Python version0 

[http : / / cars9 . uchicago . edu/software/python/mpf it .html 

• MINPACK-1 FORTRAN web page: http://netlib.org/minpack 

• MINPACK-1 pure C translation: http : //moshier . net/| 
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